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FIELD OF THE INVENTION 

The present invention relates generally to automated 
image processing and sorting, and specifically to 
automatic location of barcodes on material being sorted. 

BACKGROUND OF THE INVENTION 

Postal services and package delivery companies 
typically handle as many as several million parcels each 
day. Automated parcel sorting and routing facilities are 
being used increasingly in order to improve the 
efficiency and accuracy with which this huge volume of 
parcels is handled. 

The process of sorting and tracking parcels as they 
proceed through sorting centers requires that each parcel 
bear two types of information: the destination address of 
the parcel and a tracking number, which uniquely 
distinguishes it from other parcels in the system. The 
information is generally pr-i&fcdd on a parcel label, with 
the destination address in alphanumeric form. The 
tracking number, and frequently the address, as well, are 
printed in a machine-readable form, typically as a 
barcode. While the destination address tells where the 
parcel is to go, the tracking number assists the postal 
service or parcel company in managing its shipment 
operations and enables it to track parcels as they move 
through the system. 

In order to sort and route the parcels 
automatically, the destination address and tracking 
number are typically read by a high-resolution imaging 
system. An image processor must then rapidly locate and 
read the barcode and the destination address on the 
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parcel. This task is complicated by the fact that 
parcels vary greatly in size and shape, and may be placed 
on a conveyor belt for sorting in substantially any 
orientation. Furthermore, it frequently occurs that 
5 barcodes are located close to text and other graphic 
elements, as well as to tape or other shiny plastic items 
on the parcel, all of which add substantial "noise" to 
the barcode search. There is therefore a need for 
robust, high-speed methods that are capable of finding 
10 barcodes in a very large, noisy image within the tight 
time constraints of a large-volume package sorting 
system. 

An exemplary method for locating barcodes, 
particularly two-dimensional barcodes, is described by 

15 Wang in U.S. Patent 5,304,787, which is incorporated 
herein by reference. A stored image is processed to 
identify traversal of barcode start and stop patterns and 
to correlate these patterns with a common barcode image. 
The correlated patterns are used to identify a nominally 

2 0 rectangular area bounding the barcode in the image. The 
corners of the bounding area are used to identify the 
barcode for subsequent decoding. 
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SUMMARY OF THE INVENTION 

Preferred embodiments of the present invention 
provide improved methods and systems for rapidly locating 
barcodes and other striped structures in a large and/or 
5 noisy image. For this purpose, the image (or a region of 
interest within the image) is divided up into tiles. 
Each tile is scanned along one or more parallel lines in 
order to detect patterns of parallel stripes that could 
be indicative of the presence of a barcode in the tile, 

10 Preferably, each tile is also scanned at a diagonal to 
the original scan direction, in order to detect patterns 
of stripes that are parallel or perpendicular to the 
original scan direction and might otherwise be missed. 
Tiles that are found to contain a sufficient number of 

15 mutually-parallel stripes are tagged as likely locations 
for a barcode. The angle of the stripes in these tiles 
provides an estimate of the orientation of the barcode. 

Using this method, the entire image can be scanned 
quickly to identify all of the candidate barcode 

20 locations, without the need for an exhaustive search over 
all pixels and all possible stripe angles. The 
identification is largely insensitive to the barcode 
orientation angles and to the presence of noise and 
clutter in the vicinity of the barcodes. Based on these 

25 candidate locations and the associated orientation 
estimates, a precise orientation angle is determined for 
each barcode in the image. This angle is used to 
construct an exact rectangle that bounds the outline of 
the barcode. The barcode can then be read reliably, as 

30 well as serving as a "landmark" for finding other image 
features of importance. 
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Embodiments of the present invention are 
particularly advantageous in the context of automated 
mail processing. A parcel sorting system that uses 
automatic barcode identification for ■ recognizing and 
5 processing parcel labels is described, for example, in 
U.S. Patent Application 09/567,700, which is assigned to 
the assignee of the present patent application and is 
incorporated herein by reference. The principles of the 
present invention will also be found useful, however, in 
10 other applications in which barcodes or other striped 
structures must be identified in digital images, and 
particularly in large, noisy images. 

There is therefore provided, in accordance with a 
preferred embodiment of the present invention, a method 
15 for locating a barcode in an image, including: 

dividing the image into a plurality of tiles; 

scanning each of the tiles so as to detect a pattern 
of stripes associated with the barcode in at least one of 
the tiles; 

20 analyzing the pattern of stripes so as to determine 

an angle of orientation of the barcode; and 

responsive to the determined angle, defining bounds 
of the barcode that are aligned with the pattern of * the 
stripes . 

25 Preferably, scanning each of the tiles includes 

scanning so as to detect the pattern of stripes 
substantially irrespective of the angle of orientation of 
the barcode. Most preferably, scanning each of the tiles 
includes scanning on a first line through the tile in a 

30 first scan direction so as to detect the stripes in the 
tile that are crossed by the first line, and scanning on 
a second line through the tile in a second scan 
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direction, at a diagonal to the first direction, so as to 
detect the stripes in the tile that are crossed by the 
second line. 

Preferably, scanning each of the tiles includes 
5 detecting the pattern of stripes in a first one of the 
tiles, and defining the bounds of the barcode includes 
seeking the bounds of the barcode in at least a second 
one of the tiles, adjacent to the first one of the tiles. 

Additionally or alternatively, the stripes have 
10 respective ends, and defining the bounds of the barcode 
includes locating the ends of the stripes and delimiting 
a rectangle containing the barcode and having sides 
defined by the ends of the stripes. Preferably, defining 
the bounds of the barcode includes finding extreme lines 
15 of the barcode corresponding to first and last ones of 
the stripes of the barcode, and locating the ends of the 
stripes includes scanning along at least some of the 
stripes intermediate the first and last stripes in a 
direction parallel to the extreme lines. 
20 Preferably, the method includes reading the barcode 

responsive to the bounds. 

There is also provided, in accordance with a 
preferred embodiment of the present invention, a method 
for finding a pattern of parallel stripes in an image, 
25 which includes a plurality of pixels having respective 
pixel values, the method including: 

scanning an area of the image along a selected scan 
line in a first scan direction so as to locate a first 
sequence of the pixels on the line having pixel values 
30 within a predetermined range; 

starting from each of at least some of the pixels in 
the sequence, scanning in a second scan direction, 
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transverse to the first scan direction, to both sides of 
the line, so as to reach endpoint pixels defined by the 
first of the pixels on both sides of the line having 
pixel values outside the predetermined range; 
5 joining the endpoint pixels on each side of the line 

to define respective edges of a first one of the stripes; 

repeating the steps of scanning the area in the 
first and second scan directions and of joining the 
endpoint pixels for at least a second sequence of the 

10 pixels on the line having pixel values within the 
predetermined range, so as to define the respective edges 
of at least a second one of the stripes; and 

comparing the edges of at least the first and second 
stripes to find the stripe pattern in the image. 

15 Preferably, the method includes scanning the area 

along a further scan line, in a direction diagonal to the 
first scan direction, so as to locate third and fourth 
sequences of the pixels on the further scan line having 
pixel values within the predetermined range, and 

20 repeating the steps of scanning in the second direction, 
joining the endpoint pixels and comparing the edges with 
respect to the third and fourth sequences of the pixels 
and the diagonal direction*. - ; * 

Further preferably, "comparing the edges includes 

25 determining an orientation of the stripes in the pattern. 
Most preferably, comparing the edges includes identifying 
the stripes as belonging to the stripe pattern only if 
the edges are mutually parallel to within a predetermined 
limit. 

30 In a preferred embodiment, the pattern includes a 

barcode . 
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There is additionally provided, in accordance with a 
preferred embodiment of the present invention, apparatus 
for locating a barcode in an image, including an image 
processor, which is arranged to divide the image into a 
5 plurality of tiles, to scan each of the tiles so as to 
detect a pattern of stripes associated with the barcode 
in at least one of the tiles, to analyze the pattern of 
stripes so as to determine an angle of orientation of the 
barcode, and responsive to the determined angle, to 

10 define bounds of the barcode that are aligned with the 
pattern of the stripes. 

In a preferred embodiment, the apparatus includes an 
image capture device, which is arranged to capture the 
image of an object on which the barcode appears. 

15 Preferably, the image processor is arranged to read the 
barcode responsive to the bounds so as to extract 
information contained in the barcode, and the apparatus 
includes a sorter, which is arranged to sort the object 
responsive to the information. 

20 There is further provided, in accordance with a 

preferred embodiment of the present invention, apparatus 
for finding a pattern of parallel stripes in an image, 
which includes a plurality of pixels having respective 
pixel values, the apparatus including an image processor, 

25 which is arranged to scan an area of. the image along a 
selected scan line in a first scan direction so as to 
locate a first sequence of the pixels on the line having 
pixel values within a predetermined range, and starting 
from each of at least some of the pixels in the sequence, 

30 to scan in a second scan direction, transverse to the 
first scan direction, to both sides of the line, so as to 
reach endpoint pixels defined by the first of the pixels 
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on both sides of the line having pixel values outside the 
predetermined range, to join the endpoint pixels on each 
side of the line to define respective edges of a first 
one of the stripes, and to repeat the steps of scanning 
5 the area in the first and second scan directions and of 
joining the endpoint pixels for at least a second 
sequence of the pixels on the line having pixel values 
within the predetermined range, so as to define the 
respective edges of at least a second one of the stripes, 

10 and to compare the edges of at least the first and second 
stripes to find the stripe pattern in the image. 

There is moreover provided, in accordance with a 
preferred embodiment of the present invention, a computer 
software product for locating a barcode in an image, 

15 including a computer-readable medium in which program 
instructions are stored, which instructions, when read by 
a computer, cause the computer to divide the image into a 
plurality of tiles, to scan each of the tiles so as to 
detect a pattern of stripes associated with the barcode 

20 in at least one of the tiles, to analyze the pattern of 
stripes so as to determine an angle of orientation of the 
barcode, and responsive to the determined angle, to 
define bounds of the barcode that are aligned with * the 
pattern of the stripes. 

25 There is furthermore provided, in accordance with a 

preferred embodiment of the present invention, a computer 
software product for finding a pattern of parallel 
stripes in an image, which includes a plurality of pixels 
having respective pixel values, the product including a 

30 computer-readable medium in which program instructions 
are stored, which instructions, when read by a computer, 
cause the computer to scan an area of the image along a 
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selected scan line in a first scan direction so as to 
locate a first sequence of the pixels on the line having 
pixel values within a predetermined range, and starting 
from each of at least some of the pixels in the sequence, 
to scan in a second scan direction, transverse to the 
first scan direction, to both sides of the line, so as to 
reach endpoint pixels defined by the first of the pixels 
on both sides of the line having pixel values outside the 
predetermined range, to join the endpoint pixels on each 
side of the line to define respective edges of a first 
one of the stripes, and to repeat the steps of scanning 
the area in the first and second scan directions and of 
joining the endpoint pixels for at least a second 
sequence of the pixels on the line having pixel values 
within the predetermined range, so as to define the 
respective edges of at least a second one of the stripes, 
and to compare the edges of at least the first and second 
stripes to find the stripe pattern in the image. 

The present invention will be more fully understood 
from the following detailed description of the preferred 
embodiments thereof, taken together with the drawings in 
which : 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic, partly pictorial illustration 
of a system for sorting parcels, in accordance with a 
preferred embodiment of the present invention; 
5 Fig. 2 is a schematic representation of an image 

captured and divided into tiles for processing in the 
system of Fig. 1, in accordance with a preferred 
embodiment of the present invention; 

Fig. 3 is a flow chart that schematically 
10 illustrates a method for locating a barcode in an image, 
in accordance with a preferred embodiment of the present 
invention; 

Figs. 4A and 4B are schematic representations of a 
tile within an image of a parcel, showing details of a 
15 barcode in the tile, which is processed in accordance 
with a preferred embodiment of the present invention; 

Fig. 5 is a flow chart that schematically 
illustrates a method for detecting parts of a barcode in 
an image, in accordance with a preferred embodiment of 
20 the present invention; 

Figs. 6A and 6B are schematic representations of a 
portion of an image of a parcel, illustrating 
construction of a rectangle bounding a barcode in the 
image, in accordance with a preferred embodiment of the 
25 present invention; and 

Fig. 7 is a flow chart that schematically 
illustrates a method for constructing a rectangle that 
bounds a barcode in an image, in accordance with a 
preferred embodiment of the present invention. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Fig. 1 is a schematic, pictorial illustration of a 
system 20 for parcel sorting, in accordance with a 
preferred embodiment of the present invention. A parcel 
5 22 is transported by a conveyor 24. The parcel has a 
label 26, which typically contains a barcode 28. An 
image of the parcel, preferably a gray-scale image, is 
captured by an imaging device 30, preferably a line scan 
camera operating in a "push-broom" mode. The image 
10 captured by the camera is digitized and passed to a 
m processor 32, which applies the methods described 

y3 hereinbelow to locate barcode 2 8 on the parcel. 

^Z? Alternatively, the processor may receive the image from 

%j some other source, or it may retrieve the image from a 

15 memory (not shown) . Typically, after locating the 

= S barcode, the processor reads the barcode and address on 

!L label 26 automatically and drives a sorter 34 to route 

Iff the parcel accordingly. 

lf z Processor 32 preferably comprises a general-purpose 

p 2 0 computer, programmed with appropriate software to carry 
f ~ out the methods of the present invention. This software 

may be downloaded to the processor in electronic form, 
over a network, for example, or alternatively, it may be 
supplied on tangible media, such as CD-ROM, for 
25 installation in the processor. Such software may 

> 

similarly be adapted for use in other image processing 
applications, and may thus be supplied to and installed 
on other computers in like manner. Alternatively, the 
methods described herein may be implemented using 
30 dedicated hardware or a programmable digital signal 
processor, or using a combination of dedicated and/or 
programmable elements and/or software. The use of 
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processor 32 in the context of parcel sorting system 20 
is described here by way of example, and not limitation. 

Fig. 2 is a schematic representation of an image of 
parcel 22 captured by imaging device 30 and processed by 
5 processor 32, in accordance with a preferred embodiment 
of the present invention. This image contains two 
barcodes 28, as is typical on many of the parcels 
encountered by parcel sorting systems. In order to 
process the image,, processor 32 identifies a region of 

10 interest (ROD 36, corresponding roughly to the area of 
i parcel 22, as distinguished from conveyor 24 on which the 

parcel is resting. The ROI is divided up into tiles 38 
for further processing, wherein each of the tiles is 
preferably about 150 x 150 pixels in size. 

15 Fig. 3 is a flow chart that schematically 

illustrates a method for processing images such as that 
shown in Fig. 2 so as to find barcodes 28 in the image, 
in accordance with a preferred embodiment of the present 
invention. As noted above, processor 32 preferably finds 

20 ROI 36, at an ROI definition step 40, Alternatively, the 
method of Fig. 3 may be applied to the entire image and 
not limited to a certain ROI. The ROI is divided into 
tiles of appropriate size, at a tiling step 42. The 
processor checks each of the tiles rapidly, at a tile 
•15 selection step 44, to determine which of the tiles are 
likely to contain all or part of a barcode. Step 44 is 
based on locating parallel lines, or stripes, in the 
tile. This step is described in greater detail 

hereinbelow with reference to Figs. 4A, 4B and 5. 

30 Preferably, a tile is considered to contain a barcode (or 
part of one) if at least five such stripes are found in 
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the tile, and most of the stripes are mutually parallel 
to with about 1°. 

The orientation of the stripes that are found in a 
candidate tile provides an initial estimate of the 
orientation of the actual barcode. Processor 32 uses 
this estimate as a basis for determining the precise 
orientation angle of the barcode, at an angle calculation 
step 46. For the purpose of this step, the tile area is 
preferably binarized. Following binarization, the black 
pixels in the tile, which presumably belong to the 
barcode, are sampled, and lines are fitted through the 
sampled points at angles that are close to the estimated 
barcode orientation angle. The precise orientation angles 
of the stripes are then calculated using methods known in 
the art, such as Hough transforms. 

In order to accurately decode and use all of the 
information provided by barcode 28, it is important to 
define the bounds of the barcode precisely. For this 
purpose, processor 32 outlines a rectangular box 
containing the barcode, at a box definition step 48. 
This step is described in detail hereinbelow with 
reference to Figs. 6A, 6B and 7. If necessary, when the 
barcode extends from one tile into the next, the parts of 
the barcode in the neighboring tiles are found, and the 
box is extended accordingly. in this case, the processor 
preferably eliminates the neighboring tile or tiles from 
any further search, at a removal step 49, since the 
barcode in these tiles has already been found. 

After defining the bounding box of the barcode, the 
processor reads the barcode contents, at a reading step 
50. Additionally or alternatively, the barcode location 
and geometry are used in identifying and analyzing other 
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features in the image of parcel 22, and particularly of 
label 26, as described, for example, in the 
above-mentioned U.S. Patent Application 09/567,700 . 

Reference is now made to Figs. 4A, 4B and 5 in order 
5 to describe details of tile selection step 44, in 
accordance with a preferred embodiment of the present 
invention. Figs. 4A and 4B are schematic representations 
of an image of one of tiles 38, in which a part of 
barcode 28 is located. Fig. 5 is a flow chart that 

10 illustrates a method for rapidly scanning tile 38 to 
determine whether it contains parallel stripes that would 
qualify it as a candidate to contain the barcode. 

In Fig. 4A, three parallel vertical scan lines 56 
are defined, covering a test area 54 within tile 38. 

15 Alternatively, a greater or lesser number of the scan 
lines may be defined, and test area 54 may also be 
expanded to cover all of tile 38. Further alternatively 
or additionally, horizontal scan lines may be used for 
this purpose. At a vertical scanning step 70 (Fig. 5), 

20 processor 32 scans down a first one of scan lines 56 
until it reaches the bottom of the test area, at a 
vertical scan completion step 71. When the processor 
encounters a black pixel, at a black pixel step 72, it 
then scans transversely, to the left and right, at a 

25 transverse scanning step 73. The transverse scan is 
terminated when a white pixel is encountered on each side 
of the scan line. Typical transverse scan lines 58 
generated at step 73 are shown in Fig. 4A. 

The ends of each line 58 are marked as endpoints 60. 

30 It will be observed that for each black stripe of the 
barcode that is traversed by vertical scan line 56, the 
two sets of endpoints at either side of line 56 define 
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two line segments that run along the opposing parallel 
edges of the stripe. In Fig. 4A, only the encipoints of 
the transverse scan lines are marked, while for 
simplicity of illustration, the area between the 
5 endpoints is left blank (except in one of the scanned 
areas in which lines 58 are drawn) . Thus, processor 32 
locates stripes of the barcode by finding two opposing 
sets of endpoints 60 that define line segments that are 
parallel to within a predetermined limit, typically 1-2°. 

10 The orientation of the stripe is given approximately by 
the slopes of the line segments. As noted above, if a 
sufficient number of these stripes are found in a given 
tile, and the stripe are mutually correlated in their 
orientation angles, the processor determines that it has 

15 found a barcode. 

After scanning the first of vertical scan lines 56 
in this manner, processor 32 preferably repeats the scan 
procedure along additional scan lines in order to ensure 
that it has not missed a barcode in the tile. First, the 

20 processor goes on to scan the other vertical scan lines 
56 in substantially the same manner as it scanned the 
first line, at a vertical scan repetition step 76. Most 
preferably, the processor also performs at least one scan 
along a diagonal scan line 62 (Fig. 4B) , at a diagonal 

25 scanning step 78. As in the vertical scan, stripes of 
bar code 2 8 are found by scanning on diagonal transverse 
scan lines 64, in order to find diagonal endpoints 66. 
The diagonal scan is important in cases in which the 
stripes of the barcode are parallel to or perpendicular 

30 to vertical scan lines 56. If not for the diagonal scan, 
the processor might fail to identify the barcode in such 
a case. 
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Reference is now made to Figs. 6A, 6B and 7 in order 
to describe details of box definition step 48, in 
accordance with a preferred embodiment of the present 
invention. Figs. 6A and 6B are schematic representatives 
5 of an area of the image of parcel 22 containing barcode 
28. This area has been expanded, relative to the area of 
tile 38 shown in Figs. 4A and 4B, by adding the 
neighboring tiles so as to take in all of the barcode in 
question. Fig. 7 is a flow chart that schematically 

10 illustrates a method for determining the outline of a box 
90 (Fig. 6B) that completely contains the barcode. 

Using the barcode angle for tile 38 that was found 
at step 46 (Fig. 3), processor 32 defines a parallelogram 
80 (Fig. 6A) in which barcode 28 can be located, at a 

15 parallelogram definition step 92 (Fig. 7) . The 
parallelogram is preferably centered in the tile in which 
the barcode was identified, but it is expanded to include 
parts of the neighboring tiles. Sides 82 of the 

parallelogram are chosen to run at the determined axial 

20 angle of barcode 28 (i.e., perpendicular to the angle of 
the stripes), preferably near the edges of the barcode. 
As shown in Fig. 6A, sides 82 may still not be exactly 
aligned with the actual edges of the barcode, typically 
because of skew in the image that was originally captured 

25 by imaging device 30. 

The image within parallelogram 80 is processed to 
find the precise bounds of the barcode. Preferably, the 
image is first binarized, at a binarization step 94. 
Lines 84 at the transverse angle of barcode 28 

30 (perpendicular to sides 82) are fitted to the stripes of 
barcode 28 that fall within parallelogram 80, at a 
fitting step 96. The last line at either end of the 
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barcode is identified as an extreme line 86 (Fig. 6B) of 
the barcode, at an extremity finding step 98. The 
extreme lines are easily identified, since beyond these 
lines there is a wide extent of white space before any 
further black features are encountered. 

A subset of lines 84, preferably the lines that are 
fitted to the stripes near the center of barcode 28, are 
scanned transversely (in a direction parallel to extreme 
lines 86) to find the ends of these central lines, in a 
transverse scanning step 100. In this case, too, it is 
easy to find the ends of the lines, simply by noting the 
points at which white areas of the image are encountered 
at either end of each line. The ends of these central 
lines define the exact loci of edges 88 of the barcode. 
These edges are used to define a complete box 90 
containing the barcode, at a box completion step 102. 
The box is preferably extended slightly beyond extreme 
lines 86 in order to ensure that the information in the 
first and last stripes of-, the barcode is not lost when 
the barcode is read at step 50 (Fig. 3) . It will be 
observed that box 90 contains only the barcode of 
interest, while excluding other nearby barcode fragments 
and text. 

Although preferred embodiments are described 
hereinabove with reference certain specific methods for 
line fitting and angle determination, those skilled in 
the art will appreciate that other methods of image 
processing may also be used for these purposes. While 
the inventor has found that image binarization is useful 
in rapid processing of image data in certain of the 
processing phases described above, gray-scale and color 
image processing methods may also be used in alternative 
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embodiments of the present invention. Moreover, although 
preferred embodiments of the present invention are 
directed to finding and processing barcodes on parcels 
and other objects, the principles of the present 
5 invention may similarly be applied to detecting and 
processing other patterns of parallel lines and stripes 
that may be encountered in automated image analysis, 
including patterns appearing in binary, gray-scale and 
color images. The term "barcode" as used in the context 

10 of the present patent application and in the claims 
should therefore be taken to refer to substantially any 
machine-readable code that comprises a pattern of 
contrasting stripes . 

It will thus be appreciated that the preferred 

15 embodiments described above are cited by way of example, 
and that the present invention is not limited to what has 
been particularly shown and described hereinabove. 
Rather, the scope of the present invention includes both 
combinations and subcombinations of the various features 

20 described hereinabove, as well as variations and 
modifications thereof which would occur to persons 
skilled in the art upon reading the foregoing description 
and which are not disclosed in the prior art. 
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